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^ (54) Title: METHOD A^^:) APPARATUS FOR COORDINATING UNI>rrERRUPTroLE POV^ 
VIDE SCALABLE, REDUNDANT POWER 

ON 

(57) Abstract: Presented is a modular uninterruptible power supply system utilizing common power modules, and providing redun- 

dant coordinated control thereof. The system provides identical control logic within each of the power modules. A virtual master 
grnrn^ is established through an arbitration scheme at initialization of the UPS system. The master then assigns a virtual vice master to 

provide the redundant control should the virtual master no longer be able to perform its functions. Nearly simultaneous control of 
^5 operational mode and state change functions is accomplished through a coordinated communications system including a high-speed 

communications bus and digital control logic lines. Also provided is a mechanism for removal of the vinua] master from its duties 
>^ upon the detection of a problem that is not detected by the virtual master itself. 
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METHOD AND APPARATUS FOR COORDINATING 
UNINTERRUPTIBLE POWER SUPPLY MODULES TO PROVIDE 
SCALABLE, REDUNDANT POWER 



Field Of The Invention 
This invention relates generally to the control of uninterruptible power 
supplies, and more particularly to the control and coordination of multiple 
power modules in a modular uninterruptible power supply. 

Background Of The Tnvp»«.>» 
As more and more segments of the business environment enter the 
information age, more and more computers and computing power are 
required. As business move from the old to the new economy their reliance 
on the processing, transference, and storage of digital information is becoming 
a more and more critical aspect of their overall business strategy. While in the 
past, computer crashes were seen as a mere nuisance, the loss of computing 
power and business data may well devastate a business's ability to survive in 
today's new economy. As such, the need for reliable, uninterruptible electric 
power to maintain the operational status of the computing equipment and the 
integrity of the digital data continues to rise. 

To meet these requirements, umnterruptible power supplies (UPS) have 
been developed. These UPSs utilize a bank of electric storage batteries and 
solid state conversion equipment in association with the utility line voltage to 
provide continuous electric power to a business's computer system in the 
event of a loss or deviation of power quality from the utility. The number of 
batteries contained within an UPS is dependent upon the business's length of 
time that it needs to operate in the event of a utility power system failure. 
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Likewise, the number of power modules included in a modular UPS is - 
dependent on the overall total system load required to be supplied thereby. In 
the past, uninterruptible power supplies were only available in discreet sizes. 
However, with the recognition of the various requirements from the multitude 
5 of businesses that require UPSs, the development of modular uninterruptible 
power supplies has provided a flexibility heretofore unknown. Now, a 
businesses UPS power output may grow as its business needs by simply 
purchasing additional power modules and adding them to their existing 
modular UPS. 

10 Unfortunately, the advent of the modular UPS has also introduced new 

and different challenges to the UPS designer. Specifically, the modular UPS 
must function as a single system regardless of the number of power modules 
that are added to or taken away from the UPS chassis. Output power quality, 
system display, and transitions between operating modes must all be 

15 coordinated between the multiple power modules that make up the businesses 
UPS system. 

One such modular UPS that provides coordinated operation is 
described in U.S. Patent No. 5,982,652 to Simonelli, et al. for a METHOD 
AND APPARATUS FOR PROVIDING UNINTERRUPTIBLE POWER 

20 USING A POWER CONTROLLER AND A REDUNDANT POWER 

CONTROLLER. As described in this patent, the coordination and control oif 
the multiple power modules contained in the modular UPS is accomplished 
through the provision of a main intelligence module (MIM) and a redundant 
intelligence module (RIM). The MIM and RIM have connections to each 

25 other and to the other modules to allow monitoring of operational conditions 
of the UPS system, including characteristics of the input and output power, 
and to provide phase synchronization, frequency regulation, and voltage 
regulation of the output power. In this system, the MIM functions as the 
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prunao- controller wiflun th. UPS system and fl,e RIM is a redundant 
controller .ha. can assume con.rol of the UPS system upon failure of the MIM 
or removal of fte MD^ from fl,e UPS system. All of the power modules are 
substantially identical and perform the Actions of an unin.em.ptible power 
' supply under flie control offlje MIM or tte RIM. 

In tte system of the Simonelli, et al. '652 reference, the main 
mtelligence module (MIM) is theprima.7 computer/controller in fte UPS 
system. It acts as a central point in the system for collecting and 
communicating information about aspects ofthe power modules, banery 
n-odulcs. and fte redundant in.ellige„ce module (RIM), lie MIM contains a 
primary and a backup microprocessor for controlling and coordinating ftis 
operation. The redundant intelligence module (RIM) on the other hand is 
merely a backup ve„ion of fte MIM. andprovides redundancy in tte event of 
aMIMfailu«, orwhileaMIM is betog replaced. In this backup role a>e 
WM mcludes the same major systems as the MIM except for the main 
proce«or system, although some ofthe systems in the RIM are described as 
havmg less functionality a.an fte corresponding systems in fte MIM. 

Unfortuna.ely. such a confol sttategy greaUy increase the overall cost 
ofthe modular UPS system by requiring that two separate additional modules 
be purchased and maintained in order to allow functionality ofthe UPS at all 

In th,s way. in addition to the value providingpower modules, battery 
modules, and battety charging modules, a user is forced to also purchase a 
separate main intelligence modules to control the other components in the 
system. Further, U,e user is forced to purchase a redundant intelUgence 
module (RIM) whose only main pmpose is ,0 provide backup operation 
should the MIM fail. I„ ote words, flus module is totally superfluous to U,e 
acmal operation and functionality ofthe uninterruptible power supply unless a 
failure of the MIM occurs. 
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In an economy that demands maximum efficiency from business 
resources, such a system that requires separate primary and redundant control 
modules simply caimot be justified. The imfeasibility of such a system may 
be clearly demonstrated in a situation where a business enterprise only needs a 

5 single power module to supply its uninterruptible power requirements. In 
such a situation, the Simonelli, et ah, '652 system would require the business 
enterprise to purchase two control units, the MIM and the RIM, to control the 
single power module in its UPS. In such a situation it becomes clear that the 
provision of two control units to control a single power module is a luxury not 

10 many businesses would desire to acquire. 

Therefore, there exists a need in the art for a modular iminterruptible 
power supply that has coordinated control of its constituent components and 
an acceptable level of redundancy without requiring the purchase of additional 
multiple control units. 

« 

15 

Summary Of The Invention 

An embodiment of the invention provides a method and apparatus for 
coordinating the control of uninterruptible power supply modules to provide 
scalable, redundant power. Each of the independent UPS power modules are 

20 designed to contain identical control logic, both hardware and software, so 
that only a single power module need be purchased to scale the UPS system 
based on the business enterprises ppwer requirements. Each of these identical 
UPS power modules are connected to common communications, power, and 
logic buses within the UPS system. Under such a system, the multiple power 

25 modules act as a single UPS with all of the controls and actions taken by the 
individual power modules being coordinated through a single power module 
acting as the system's virtual master. Preferably, the master role is virtual 
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because any of the power modules in the system contain all of the necessary 
control logic and software to assume the master role. 

For the system of the invention to be fault tolerant it must be capable of 
withstanding a failure or removal of the power module that has assumed the 
5 master role. Preferably, this is accomplished by requiring another module to 
be prepared to provide backup to the master. In this role, designated vice- 
master, the power module provides the controlling signals to itself and the 
other power modules if and when the designated master fails. Under this 
system, the vice-master then promotes itself to master and assigns ^ new vice- 
master from one of the operational power modules in the system. Under the 
system of the invention, no additional main intelligence modules or redundant 
mtelligence modules are required to perform this functionality. Coordination 
and redundancy are provided within the control of the mdividual power 
modules themselves. 

A power up arbitration scheme is used to assign one of the power 
modules to the master's role. Preferably, the master then assigns a vice-master 
and assigns all other modules to act as peers. Peer modules simply follow the 
control commands of the master or vice-master depending on the operational 
. status of these two units. Modules itf =fhe peer role are also drawn upon when 
a vice-master needs to be appointed to the master role upon detection of a 
problem with that unit. Hiis hierarchy of roles provides the means to 
coordinate identical UPS modules intp a larger UPS system without the need 
for additional special main and backup controllers. In the event that only a 
single power module is comiected in the UPS system, it is quite capable of 
controlling itself in the role of the master, without a requirement for a vice- 
master or any peer modules to be connected. Under the system of the 
invention, there is no need for two separate control modules to control the one 
power module connected in the system. 
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Power module interconnection and coordination of control is 
accomplished using a common set of signals, collectively known as the 
redundancy management bus (RMB). The RMB of the invention includes a 
high-speed communication bus, a set of bi-directional logic lines, and one 
5 analog reference for each power phase of the UPS system. Each power 

module may drive any of the signals. It is the role of the power module that 
determines which signals are driven and which signals are merely monitored. 
The high-speed communication bus may take any one of a number of forms, 
including CAN, Ethernet, Serial 485, etc. In a preferred embodiment, a high- 

10 speed communication bus takes the form of a CAN bus due to CAN's inherit 
fault tolerance and noise immunity. 

In accordance with the invention, the master is responsible for driving 
most of the signals on the RMB. The other power modules take direction 
from the master by acting on these signals. When the master power module 

15 fails or is removed from the UPS system, a master ok signal typically supphed 
by the master power module is no longer sent. This provides an indication 
that the vice-master power module should take over driving the analog output 
voltage target signal used by the power modules to create their output voltage 
waveforms. Since the role of the vice-master has previously been assigned, 

20 the remaining peers do not see a discontinuity in the output voltage target 
signal and therefore their inverter output waveforms are clean during this 
transition. 

In a preferred embodiment of the invention, the microprocessor inside 
each of the power modules is connected to the RMB through an interface 
25 computer programmable logic device (CPLD). Preferably two digital signals 
from the processor are used to indicate the power module's role to the CPLD. 
The role logic is used to enable bus drivers for each role's group of output 
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signals. The power module's particular role in a given UPS system 
determines the bus input/output direction for each communicated signal. 

Other objects and advantages of the invention will become more 
apparent from the following detailed description when taken in conjunction 
with the accompanying drawings. 



a 



Brief Descrip tion Of Thp Drawin; 
The accompanying drawings incorporated in and forming a part of the 
specification, illustrate several aspects of the present invention, and together 
with the description serve to explain the principles of the invention. In the 
drawings: 

FIG. 1 is a system level single line diagram of a modular 
uninterruptible power supply system constructed in accordance with the 
teachings of the invention; ■ 

FIG. 2 is a detailed single line block diagram of the control structure of 
power module constructed in accordance with the teachings of the invention; 
FIG. 3 is a control flow diagram illustrating the operating role 
determination process of the invention; 

FIG. 4 is a control flow diagram illustrating master control of a 

configuration state change of the individual power modules in the UPS of 
FIG. 1; . 

FIG. 5 is a control flow diagram illustrating a control methodology to 
allow power modules in the UPS system of FIG. 1 to vote out a master 
controller; 

FIG. 6 is a control flow diagram illustrating the corresponding 
functional control logic within the virtual master in response to the vote out 
control flow of FIG. 5; and 
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FIG. 7 is a control flow diagram illustrating the assumption of the 
virtual master role by a virtual vice master in accordance with the teachings of 
the invention. 

While the invention will be described in connection with certain 
5 preferred embodiments, there is no intent to limit it to those embodiments. On 
the contrary, the intent is to cover all alternatives, modifications and 
equivalents as included within the spirit and scope of the invention as defined 
by the appended claims. 

10 Detailed Description Of The Preferred Embodiments 

A preferred embodiment of the modular uninterruptible power supply 
(UPS) 10 constructed in accordance with the teachings of the invention is 
illustrated in single line block diagrammatic form in FIG. 1. As with typical 
modular UPS's, the modular UPS 10 of the invention includes multiple slots 

1 5 22a-f into which various modular components may be placed. As illustrated 
in FIG. 1, modules 22a-c have installed therein power modules 12a-c 
constructed in accordance with the teachings of the invention. Slots 22d-f are 
illustrated in FIG, 1 as being empty, thus providing the ability to expiand or 
vary the power generation capability of the modular UPS 10. 

20 The AC input power from the utility is connected to the UPS 1 0 

through lines 16, and the output of the UPS 1 0 is supplied to the utilization 
equipment through output lines 18/ Preferably, each of the power modules 
12a-c are identically constructed, and contain a power inverter 20a-c and a 
controller 24a-c. As will be recognized by those skilled in the art, the 

25 particular topology utilized for the construction of the inverter 20 may vary 
substantially based on system and performance level requirements. 
Preferably, the inverter 20 utilizes a double conversion H-bridge with its own 
internal bypass. The control for this inverter 20 is provided by the controller 
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24 through the control lines 32a-c. This control accomplishes both the proper 
operation of the power switches of the inverter to construct the output AC 
wave form, as well as the selection and designation of the proper operating 
mode for the inverter. These operating modes may include a bypass mode of 
operation for high efficiency or fault tolerance, and two modes of inverter 
operation. The furst allowmg the AC voltage from the utility to supply the 
inverter, while the second utilizing installed battery banks (not shown) to 
power the inverter. 

As discussed above, the need for coordinated control between the 
individual power modules is paramount so that the UPS 1 0 may operate as an 
integrated unit. This unified and integrated control is accomplished by the 
separate control modules 24a-c in each of the individual, identical power 
modules 12a.c. The controllers 24a-c each receive operating parameters of its 
own power module, such as input power parameters via line 26c and output 
15 power parameters via line 28c. Additionally, the control modules 

communicate with one another via a high-speed communications bus 34. This 
communications bus may be constructed in accordance with any of a number 
of high-speed communication standards, including CAN, Ethernet, serial 485, 
etc. Preferably, this bus is implemented as a CAN bus in the UPS system 10 
20 due to CAN'S inherent fault tolerance and noise immunity. 

In addition to the high-speed communication, and for reasons which 
will be discussed more fiilly below.the controUers 24a-c also communicate 
via a set of bi-directional logic lines 36. Further, communication between 
each of the controllers 24a-c to aUow proper inverter coordination is 
accomplished via an analog reference .38 for each power phase of the UPS 
system 10. Collectively, this common set of signals is known as the 
redundancy management bus (RMB). Advantageously, each of the power 
modules 12a-c are capable of driving any of the signals contained on the RMB 



25 
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based upon an established role assumed by the power module 12a-c as will be 
discussed below. This role of the power module determines which signals are 
driven and which signals are monitored. 

FIG. 2 illustrates an individual power module 12 in greater detail. As 
may be seen from this detailed illustration, the controller 24 includes control 
logic 40 which is responsible for coordinating the* integrated control of this 
particular power module 12 with the others included in the UPS system. The 
control logic 40 communicates via the bi-directional communication bus 34 to 
send and receive status information and nontime critical control signals. The 
control logic 40 may also receive a position signal 42 that designates in which 
slot of the UPS chassis the power module 12 is installed. This signal may also 
come from a user interface display controller that drives a user interface for 
the UPS, if one is included. The control logic 40 also communicates through 
a computer programmable logic device (CPLD) 44 over the set of bi- 
directional logic lines 36. The CPLD is utilized to properly configure the 
control signals via lines 36 in input or output mode for communication to the 
control logic 40 via lines 46 based on the role that the control logic 40 has 
assumed in the overall system. This role is communicated to the CPLD 44 via 
the role selection lines 48. 

As will be discussed in greater detail below, the control logic 40 may 
determine that its power module 12 should be in the role of mastei-, vice 
master, or peer. While various control signals may be utilized, a preferred 
embodiment of the invention contains control signals as indicated in Table 1. 
This table also indicates the direction of signal transfer depending on the role 
assumed by the control logic 40, The role selection signals 48 for this 
embodiment are indicated in Table 2. Table 3 defines a preferred pin out 
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arrangement for the CPLD 44 to allow proper coordination of the input and 
output information conveyance based upon the role assumed by the particular 
power module 12. 



5 Table 1 



RMB Signal 


" lO 
Diretion on 
Master 


' lO 
Direction 
on Vice- 
Master 


■ ID 
Direction 
on Peer 


Description ' ** — 


MasterOK 


Output 


Input 


Input 


Indicates the master is functioning 
properly from its perspective. Its 
Watchdog and other checks are normal. 
When this signal is dropped, ViceMaster 
takes over driving the Output signals. 


ViceMasterOK 


Input 


Output 


Input 


Indicates the vise-master is functioning 
properly from its perspective. 


VMVoteOut 


Input 


Output 


Input 


Vice-Master asserts this line and requests 
that the current master be voted out of 
office. This is done when errors are seen 
oy me vice-Master m the cross channel 
data." 


PeerVoteOut 


Input 


Input 


Output 


Any non-Master or non- Vice-Master 
asserts this luxe and requests that the 
current master is voted out of ofQce. 
This is done when errors are seen by the 
Peer in the cross channel data. 


Start " 


Output 


Input . 


Input 


Controlled by the Master, this signal 
forces the previously requested action to 
begin on each of the functional peers. It 
is tied to an ISR on the power module 


Line Loss 


Output 


Input 


Input 


Signals an input line has been lost and 
the ups should go on inverter. 


Verror 


Analog 
output 


Analog 
input (output 
on master 
failure) 


Analog 
input 


This IS the output voltage target, which is 
set by the master and used by all power 
modules in the system. One for each line 
phase is needed. 


CAN 


Both 


Both 


^Both 


All modules may communicate on the 
CAN bus. 
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Table 2 



Role/Enable Group 


Drive Enable Equation 


Description 


Master Output 
Enable (M_out-en) 


= Ready Sl WDOK & i VoteOut & Master 
& IVmaster 


Acting as Master - conditionally 
drives the Master's Output lines, • 
All other lines are inputs to 
CPU. 


Vice-Master Output 
Enable (VM_out_en) 


= Ready & WDOK & VoteOut & 'Master 
and Vm aster 


Acting as ViceMaster - 
conditionally drives the 
ViceMaster's Output lines. All 
other lines are inputs to CPU. 


Peer Output Enable 
(P_put_en) 


= Ready & WDOK & (Master and IVmaster 


Acting as Peer - may drive the 
PeerVoteOut request. All other 
lines are input to CPU. 


Startup 


= Master & Vm aster 


Startup condition — All outputs 
are high impedance. Ready 
latch state is set. On power-up 
this is the starting point. All 
lines are inputs to CPU. 


Verror Drive 


= M_out_en + (VM_out_en & !MasterOK) 


Control's the driving of the 
analog Verror signal on the bus. 



Table 3 



Signal Name | CPLD Assignment/Equations 


Description 


LocalOK 


Pin on CPU side - 1 (input to CPLD) 


Local Controller software 
believes it is functioning 
properly. Output from CPU. 


Master 


Pin on CPU side -I 


Indicates if controller is in 
Master or Slave mode. Output 
from CPU. 


Vmaster 


Pin on CPU side -I 


Indicates if controller is in 
Vice-Master or Slave mode. 
Output from CPU. 


WDOK 


Pin on CPU side - 1 


Local Watchdog monitor is OK 
and is being serviced. Output 
from CPU. 


RdyClock 


Input clock pin 


Input clock for Ready state. 




Bus Side 




MasterOK_bp 


= Master & IVmaster & LocalOK & Ready OE 
= M_out_en 


Pin on Bus side - I/O (both 
Input/Output) Bus drive or 
input based on Out enable. 


ViceMasterOK^b 
P 


= Vmaster & 'Master & LocalOK 8c Ready OE 
= VM out en 


Pin on Bus side - VO Bxis drive 
on input based on Out enable. 


VMVoteOut_bp 


= VMVoteOut_c & vmaster OE = VM_out_en 


Pin on Bus side - I/O Bus drive 
or input based on Out enable. 


PeerVoteOut_bp . 


= PeerVoteOut_c & peer OE = VM_out_en 


Pin on Bus side - I/O Bus drive 
or input based on Out enable. 


Start^bp 


= Startle OE - M_out_en 


Pin on Bus side - I/O Bus drive 
on input based on Out enable. 


LineLoss_bp 


= LineLoss_c OE = M_out_en 


Pin on Bus side - I/O Bus drive 
or input based on Out enable. 


Verr Drive 


= M_out_en + (IMasterOKJ) & VM_out_en) 
OE = M out en 


Pin on bus side - O Enables 
analog drive. 



wo 01/73921 



PCT/USOl/09722 



13 



MasterOK_c 


CPU Side 

= MasterOK_bp - M_out_en (if master) 

= ViceMasterOK^bp = fM_out_en (if vice- 
master) 


Pin on CPU ciHp n Tn/i:A<i.«>^. 
Master is ok or not. 
Pin on CPU side - O Indicates " 
VfceMaster is ok or not. 


YiceMasterOK_c 


VMVoteOut_c 
TeerVoteOuTc " 


- VMVoteOut^bp (if master) OE=! VM_out_en 


Pin on CPU side - I/O I/O for ~ 
CPU based on Out enable. 




= PeerVoteOut_bp (if master) OE=IP_out_en 


Pin on CPU side - I/O I/O for 
CPU based on Out enable 


Startle 


= Start^bp (if !master) OE = !M_out_en 


Pin on CPU side - I/O I/O for 
CPU based on Out enable 


LineLoss^c 


LineLoss^bp (if Imaster) OE = !M_out_en 


Pin on CPU side I/O I/O for " 
CPU based on Out enable. 



As may further be seen in FIG. 2, the controller 24 also, includes ari 
inverter control section 50 which operates to control the inverter based upon 
inputs 52 from the control logic 40, input and output power monitoring signals 
26, 28, as well as a voltage reference signal 54 from the analog reference line 
38. This analog voltage reference line 38 may be driven by this particular 
power module 12 depending on the role assumed by the control logic 40. 
That is, if the control logic 40 has assumed the role of master, the CPLD 44 
will enable 56 an analog voltage reference module 58 to generate and transmit 
the voltage reference signal on the analog voltage reference line 38. In the 
case where this power module 12 has assumed the role of master, its own 
analog voltage reference circuitry 58 will be driving the inverter control 50. 
For modules that have assumed roles other than the master, their analog 
voltage reference circuitry 58 ^yill.not be enabled via line 56 from the CPLD 
44, and therefore the voltage reference signal 54 will be that determined by 
another power module. 

The determination of the proper role to be assumed by the control 40 of 
each individual power module 12 is determined initially at power up, as 
illustrated in flow diagrammatic form in FIG. 3. Once the control logic 
powers up 60 it checks the master okay signal to detemiine if any other units 
have asserted the master role 62. This is allowed at power up because the 
CPLD 44 (see FIG. 2) initially configures the control lO lines 46 in a receive 
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mode until a role is established via the role selection lines 48. If another unit 
has already asserted the role of master, as may be the case if a power module 
is inserted to the UPS. during operation, the control logic 40 will request a role 
64 from the master. This role request signal is preferably transmitted via the 
5 high-speed bi-directional communications bus 34. Once the master has 
determined a role for this particular power module, it will receive this 
information 66 and thereafter properly configure its conununications for the 
assigned role 68 before terminating this process 70. As discussed above, the 
configuration of the communications based on the role assigmnent allows the 
10 bi-directional signal lines to be properly interpreted as inputs or outputs in 
accordance with the direction information contained above in Table 1. 

If, however, upon power up 60 it appears that no other power module 
has asserted the master role 62, this power module will begin to perform, its 
master arbitration 72. This master arbitration is designed to bias a race 
15 between identical power modules so that only one winner attempts to assert 
the master role. Known methods for this arbitration include the generation of 
a random number within the power module. This random number is then used 
as a multiplier for a fixed time delay which must time out prior to the 
attempted assertion of the master role. In a preferred embodiment, this master 
20 arbitration utilizes position information conmiunicated to the control logic 40 
either from physically sensing the particular slot in which it is installed, or via 
a location signal transmitted from a common user interface and display 
controller if such is provided in the UPS 1 0. 

Once the power module has completed its master arbitration 72, it 
25 checks to determine if any other control unit has already asserted the master 
role 74. If another unit has asserted the master role, this power module 
transmits a role request 64 to the master so that it may be assigned its proper 
role in the UPS configuration- Once it receives its role 66 it will then 
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configure its communications for that role 68 before ending this process 70. 
If, however, no other power module has yet asserted the master role, the 
control logic 40 will assert that it is the master 76. Once this assertion has 
been made, the power module will then configure its communications for its 
5 role 68 before ending this process 70, 

While in the role of master, that power module is responsible for 
controlling the state configuration of the entire UPS. When a state change is 
required, it is the responsibility of the master to coordinate this state change so 
that all power modules will perform the state change transitions at nearly L 
identical time. This coordination through the master unit is illustrated in the 
simplified flow diagram of FIG. 4. Once started 78 the master then 
determines that a configuration state change is needed 80. This determination 
may be based on a number of different factors, including user input, 
operational status of the power generation and distribution, fault detection, 
etc. Once the new configuration is determined by the master, it then transmits 
the new configuration information to the vice master and the peer power 
modules in the UPS system 82. This infonnation is preferably transmitted via 
the high-speed communication bus 34 (see FIG. I) to all of the other power 
modules. This information is used by the other power modules to prepare for 
the state change to minimize the system disturbance resulting therefrom. 

Once the new configured information has been transmitted to all of the 
other power modules in the UPS system, the master then toggles 84 one of the 
control Imes known as the synch line to initiate the state change. The sync 
line preferably initiates a high-level or non-maskable interrupt in controller 40 
of each of the power modules in the UPS system, including itself. The high 
level interrupt is then processed and the units perform the state change 86 
before ending the sequence 88. This control structure is utilized in a preferred 
embodiment to allow for nearly simultaneous transition between operating 
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modes on each of the individual power modules in the system. Controlling 
these state changes requires tight timing on the execution skew between these 
distributed power modules, and the synch line and the high-speed 
communication bus are used in combination to accurately synchronize actions 
5 and events between the collection of the uninterruptible power supply 
modules. 

While the high-speed communication bus provides an excellent means 
of triggering actions, microsecond synchronization of actions on multiple 
different processors is difficult because of the asynchronous nature of the 

10 communications. Communication latencies and interrupt delays limit the 

accuracy to only a few milliseconds. While such delays may be acceptable in 
systems utilizing ferroresonant transformer coupled outputs, such delays may 
be catastrophic in a transformerless system such as is used in a preferred 
embodiment of the invention. By utilizing the synch line and tying it to a high 

15 level interrupt for each of the controllers, the communication variability is 

removed. By processing the state change as a high level interrupt, the actions 
across the multiple controllers for each of the multiple power modules are all 
started at approximately the same time within the limits of the processor's 
high level interrupt handling. In a preferred embodiment of the invention, this 

20 is typically a few tenths of microseconds, and s in any event, much tighter 
than the communications bus would allow if it were used independently. 

In addition to providing redundancy through the role assumption 
scheme discussed above, the system of the invention also includes a method 
for removing the current master in the event that other units sense a problem 

25 that the master cannot sense for itself. As indicated in the flow diagram of 
FIG. 5, once this process is started 90 and the unit senses any type of problem 
with the master 92 that qualifies as a removable problem, the imit transmit a 
vote out signal 94 to the master before ending 96. Qualifying problems are 
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defined by the system requirements, and may include communication failures 
with the master, load sharing and balance problems due to a problem in the 
master's transmission of the Ven^, signal, etc. 

Within the master, as illustrated in the flow diagram of FIG. 6, once 
5 this process is started 98, the power module checks to determine that it is.the 
master 100. If the power module is not in the role of master, the process ends 
102. However, if the power module is in the role of master, it first checks to 
determine if it has received a vote out signal from its vice master 104. If it has 
received the vice master vote out signal 104 it then checks to determine if it 
10 has received a vote out signal from at least one peer 106 in the UPS system. If 
the master has not received a vote out signal from two units, one being the 
vice master and one being at least one of the peers, the master does not 
relinquish its role as master. I^ however, the master has received a vote out 
signal from both the vice master and one of the peers, it then drops its master 
1 5 role assertion 108 before ending this process 1 02. 

Once the master has dropped its assertion of the master role, it is then 
the duty of the vice master to assume those responsibilities as illustrated in the 
flow diagram of FIG. 7. Upon initialization of this process 1 10 the power 
module checks to determine if its role has been assigned as the vice master 
112. Ifnot, this process ends 114 and the power module continues as a peer in 
the UPS system. However, for the power module that has assumed the role of 
vice master, it continuously checks to make sure that the master is still okay 
1 16. As soon as it is determined that the master is no longer okay, the vice 
master then begins to transmit the signal 1 1 8 so that there is no, or only a 
25 minimal disturbance in this signal to the other power inverters. The vice 
master then asserts the master role 120 and reconfigures its communications 
for the master role 1 22 . 
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Once this has been accomplished, the new master then chooses one of 
the peer modules as the vice master and transmits its new assignment to that 
power module 124 before ending this process 1 14. The assignment of the vice 
master may be accomplished through any number of control strategies from 
5 the available peers in the UPS system. Preferably, the vice master simply 
picks one of the peers based on a location priority and transmits the new role 
assignment thereto. Alternatively, the new master could transmit a master 
arbitration signal to each of the peers in the UPS system and assign the vice 
master based upon the winner of that arbitration. 

10 The foregoing description of various preferred embodiments of the 

invention has been presented for purposes of illustration and description. It is 
not intended to be exhaustive or to limit the invention to the precise forms 
disclosed. Obvious modifications or variations are possible in light of the 
above teachings. The embodiments discussed were chosen and described to • 

15 provide the best illustration of the principles of the invention and its practical 
application to thereby enable one of ordinary skill in the art to utilize the 
invention in various embodiriients and with various modifications as are suited 
to the particular use contemplated. AH such modifications and variations are 
within the scope of the invention as determined by the appended claims when 

20 interpreted in accordance with the breadth to which they are fairly, legally, 
and equitably entitled. 
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What Is ClaimftH Tc - 

1 . A redundant modular uninterruptible power supply system, 
comprising: 

a chassis defining a plurality of slots adapted to accommodate a 
plurality of power modules therein, said chassis further providing input power 
lines and output power lines to each of said slots; and 

a plurality of power modules adapted to mount in said slots, said power 
modules including a power inverter coupled to said input power linfes and to 
said output power lines, and a controller, said controller being operably 
coupled to said power inverter to control operation thereof; and 

wherein one of said controllers in one of said plurality of power 
modules asserts itself as virtual master to coordmate operation of all power 
modules; and 

wherein said virtual master appoints one of said controllers in one of 
. said plurality of power modules as a vice master to provide redundancy of 
control in the event said virtual master fails. 

■ 2. A method of providing redundant, coordinated control of a 
modular uninterruptible power supply having multiple, identical, power 
modules included therein, each of the power modules including a controller, 
comprising the steps of; 

performing master arbitration between said controllers of said multiple, 
identical power modules; 

asserting one of said controllers as a virtual master to coordinate 
operation of the modular uninterruptible power supply; 

assigning one of the remaining controllers as a vice master to provide 
redundancy of control in the event the virtual master fails. 
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